{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "e4cc07ed",
      "metadata": {
        "id": "e4cc07ed"
      },
      "source": [
        "#  Introduction To Material Science\n",
        "\n",
        "## Table of Contents:\n",
        "* [Introduction](#introduction)\n",
        "* [Setup](#setup)\n",
        "* [Featurizers](#featurizers)\n",
        "    - [Crystal Featurizers](#crystal-featurizers)\n",
        "    - [Compound Featurizers](#compound-featurizers)\n",
        "* [Datasets](#datasets)\n",
        "* [Predicting structural properties of a crystal](#pred-props)\n",
        "* [Further Reading](#further-reading)\n",
        "\n",
        "## Introduction <a class=\"anchor\" id=\"introduction\"></a>\n",
        "\n",
        "One of the most exciting applications of machine learning in the recent time is it's application to material science domain. DeepChem helps in development and application of machine learning to solid-state systems. As a starting point of applying machine learning to material science domain, DeepChem provides material science datasets as part of the MoleculeNet suite of datasets, data featurizers and implementation of popular machine learning algorithms specific to material science domain. This tutorial serves as an introduction of using DeepChem for machine learning related tasks in material science domain.\n",
        "\n",
        "Traditionally, experimental research were used to find and characterize new materials. But traditional methods have high limitations by constraints of required resources and equipments. Material science is one of the booming areas where machine learning is making new in-roads. The discovery of new material properties holds key to lot of problems like climate change, development of new semi-conducting materials etc. DeepChem acts as a toolbox for using machine learning in material science.\n",
        "\n",
        "This tutorial can also be used in Google colab. If you'd like to open this notebook in colab, you can use the following link. This notebook is made to run without any GPU support.\n",
        "\n",
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/deepchem/deepchem/blob/master/examples/tutorials/Introduction_To_Material_Science.ipynb)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "b3a2e5d4",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "b3a2e5d4",
        "outputId": "f07fa8ce-3f57-4e0d-e1c5-d14cdbdd3989"
      },
      "outputs": [],
      "source": [
        "!pip install --pre deepchem"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "cbec9313",
      "metadata": {
        "id": "cbec9313"
      },
      "source": [
        "DeepChem for material science will also require the additiona libraries [`pymatgen`](https://pymatgen.org/) and [`matminer`](https://hackingmaterials.lbl.gov/matminer/). These two libraries assist machine learning in material science. For graph neural network models which we will be used in the backend, DeepChem requires [`dgl`](https://www.dgl.ai/) library. All these can be installed using `pip`. **Note** when using locally, install a higher version of the jupyter notebook (>6.5.5, here on colab)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "id": "f162536e",
      "metadata": {
        "id": "f162536e"
      },
      "outputs": [],
      "source": [
        "!pip install -q pymatgen==2023.12.18\n",
        "!pip install -q matminer==0.9.0\n",
        "!pip install -q dgl\n",
        "!pip install -q tqdm"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "3335e96f",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 288
        },
        "id": "3335e96f",
        "outputId": "7c898cfc-a9dc-4f07-8b8d-ee290b493a8c"
      },
      "outputs": [],
      "source": [
        "import deepchem as dc\n",
        "dc.__version__"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "id": "6f76f7a5",
      "metadata": {
        "id": "6f76f7a5"
      },
      "outputs": [],
      "source": [
        "from tqdm import tqdm\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "import pymatgen as mg\n",
        "from pymatgen import core as core\n",
        "\n",
        "import os\n",
        "os.environ['DEEPCHEM_DATA_DIR'] = os.getcwd()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "d43c4bc3",
      "metadata": {
        "id": "d43c4bc3"
      },
      "source": [
        "## Featurizers <a class=\"anchor\" id=\"featurizers\"></a>\n",
        "\n",
        "### Material Structure Featurizers <a class=\"anchor\" id=\"crystal-featurizers\"></a>\n",
        "\n",
        "Crystal are geometric structures which has to be featurized for using in machine learning algorithms.  The following featurizers provided by DeepChem helps in featurizing crystals:\n",
        "\n",
        "- The [SineCoulombMatrix](https://deepchem.readthedocs.io/en/latest/api_reference/featurizers.html#sinecoulombmatrix) featurizer a crystal by calculating sine coulomb matrix for the crystals. It can be called using `dc.featurizers.SineCoulombMatrix` function. [1]\n",
        "- The [CGCNNFeaturizer](https://deepchem.readthedocs.io/en/latest/api_reference/featurizers.html#cgcnnfeaturizer) calculates structure graph features of crystals. It can be called using `dc.featurizers.CGCNNFeaturizer` function. [2]\n",
        "- The [LCNNFeaturizer](https://deepchem.readthedocs.io/en/latest/api_reference/featurizers.html#lcnnfeaturizer) calculates the 2-D Surface graph features in 6 different permutations. It can be used using the utility `dc.feat.LCNNFeaturizer`. [3]\n",
        "\n",
        "[1] Faber et al. “Crystal Structure Representations for Machine Learning Models of Formation Energies”, Inter. J. Quantum Chem. 115, 16, 2015. https://arxiv.org/abs/1503.07406\n",
        "\n",
        "[2] T. Xie and J. C. Grossman, “Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties”, Phys. Rev. Lett. 120, 2018, https://arxiv.org/abs/1710.10324\n",
        "\n",
        "[3] Jonathan Lym, Geun Ho Gu, Yousung Jung, and Dionisios G. Vlachos, Lattice Convolutional Neural Network Modeling of Adsorbate Coverage Effects, J. Phys. Chem. C 2019 https://pubs.acs.org/doi/10.1021/acs.jpcc.9b03370"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "cbfd1c09",
      "metadata": {
        "id": "cbfd1c09"
      },
      "source": [
        "#### Example: Featurizing a crystal\n",
        "\n",
        "In this part, we will be using `pymatgen` for representing the crystal structure of Caesium Chloride and calculate structure graph features using `CGCNNFeaturizer`.\n",
        "\n",
        "The `CsCl` crystal is a cubic lattice with the chloride atoms lying upon the lattice points at the edges of the cube, while the caesium atoms lie in the holes in the center of the cubes. The green colored atoms are the caesium atoms in this crystal structure and chloride atoms are the grey ones.\n",
        "\n",
        "<img src=\"https://github.com/deepchem/deepchem/blob/master/examples/tutorials/assets/CsCl_crystal_structure.png?raw=1\">\n",
        "\n",
        "Source: [Wikipedia](https://en.wikipedia.org/wiki/Caesium_chloride)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "id": "592737b4",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "592737b4",
        "outputId": "0237d60c-ee21-4b54-ce63-c245d30261e8"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Structure Summary\n",
              "Lattice\n",
              "    abc : 4.2 4.2 4.2\n",
              " angles : 90.0 90.0 90.0\n",
              " volume : 74.08800000000001\n",
              "      A : 4.2 0.0 0.0\n",
              "      B : 0.0 4.2 0.0\n",
              "      C : 0.0 0.0 4.2\n",
              "    pbc : True True True\n",
              "PeriodicSite: Cs (0.0, 0.0, 0.0) [0.0, 0.0, 0.0]\n",
              "PeriodicSite: Cl (2.1, 2.1, 2.1) [0.5, 0.5, 0.5]"
            ]
          },
          "execution_count": 5,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# the lattice paramter of a cubic cell\n",
        "a = 4.2\n",
        "lattice = core.Lattice.cubic(a)\n",
        "\n",
        "# Atoms in a crystal\n",
        "atomic_species = [\"Cs\", \"Cl\"]\n",
        "# Coordinates of atoms in a crystal\n",
        "cs_coords = [0, 0, 0]\n",
        "cl_coords = [0.5, 0.5, 0.5]\n",
        "structure = mg.core.Structure(lattice, atomic_species, [cs_coords, cl_coords])\n",
        "structure"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5ab7f770",
      "metadata": {
        "id": "5ab7f770"
      },
      "source": [
        "In above code sample, we first defined a cubic lattice using the cubic lattice parameter `a`. Then, we created a structure with atoms in the crystal and their coordinates as features. A nice introduction to crystallographic coordinates can be found [here](https://www.youtube.com/watch?v=dP3LjWtoeMU). Once a structure is defined, it can be featurized using CGCNN Featurizer. Featurization of a crystal using `CGCNNFeaturizer` returns a DeepChem GraphData object which can be used for machine learning tasks."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "id": "c31ebc0a",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "c31ebc0a",
        "outputId": "da508e82-fd31-4162-ed2d-5fd81479c52c"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "GraphData(node_features=[2, 92], edge_index=[2, 24], edge_features=[24, 41])"
            ]
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "featurizer = dc.feat.CGCNNFeaturizer()\n",
        "features = featurizer.featurize([structure])\n",
        "features[0]"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "8f7e8c58",
      "metadata": {
        "id": "8f7e8c58"
      },
      "source": [
        "### Material Composition Featurizers  <a class=\"anchor\" id=\"compound-featurizers\"></a>\n",
        "\n",
        "The above part discussed about using DeepChem for featurizing crystal structures. Here, we will be seeing about featurizing material compositions. DeepChem supports the following material composition featurizers:\n",
        "\n",
        "- The [ElementPropertyFingerprint](https://deepchem.readthedocs.io/en/latest/api_reference/featurizers.html#molecule-tokenizers) can be used to find fingerprint of elements based on elemental stoichiometry. It can be used using a call to `dc.featurizers.ElementPropertyFingerprint`. [4]\n",
        "- The [ElemNetFeaturizer](https://deepchem.readthedocs.io/en/latest/api_reference/featurizers.html#elemnetfeaturizer) returns a vector containing fractional compositions of each element in the compound. It can be used using a call to `dc.feat.ElemNetFeaturizer`. [5]\n",
        "\n",
        "[4] Ward, L., Agrawal, A., Choudhary, A. et al. A general-purpose machine learning framework for predicting properties of inorganic materials. npj Comput Mater 2, 16028 (2016). https://doi.org/10.1038/npjcompumats.2016.28\n",
        "\n",
        "[5] Jha, D., Ward, L., Paul, A. et al. \"ElemNet: Deep Learning the Chemistry of Materials From Only Elemental Composition\", Sci Rep 8, 17593 (2018). https://doi.org/10.1038/s41598-018-35934-y\n",
        "\n",
        "#### Example: Featurizing a compund\n",
        "\n",
        "In the below example, we featurize Ferric Oxide (Fe2O3) using `ElementPropertyFingerprint` featurizer . The featurizer returns the compounds elemental stoichoimetry properties as features."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "id": "5c7ed055",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5c7ed055",
        "outputId": "fa1d6652-4a4b-475f-dbd8-b0258d8cc8c2"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/pymatgen/core/periodic_table.py:186: UserWarning: No data available for electrical_resistivity for O\n",
            "  warnings.warn(f\"No data available for {item} for {self.symbol}\")\n",
            "/usr/local/lib/python3.10/dist-packages/pymatgen/core/periodic_table.py:186: UserWarning: No data available for bulk_modulus for O\n",
            "  warnings.warn(f\"No data available for {item} for {self.symbol}\")\n",
            "/usr/local/lib/python3.10/dist-packages/pymatgen/core/periodic_table.py:186: UserWarning: No data available for coefficient_of_linear_thermal_expansion for O\n",
            "  warnings.warn(f\"No data available for {item} for {self.symbol}\")\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "array([1.83000000e+00, 3.44000000e+00, 1.61000000e+00, 2.79600000e+00,\n",
              "       1.13844192e+00, 2.00000000e+00, 4.00000000e+00, 2.00000000e+00,\n",
              "       2.80000000e+00, 1.41421356e+00, 8.00000000e+00, 1.60000000e+01,\n",
              "       8.00000000e+00, 1.28000000e+01, 5.65685425e+00, 2.00000000e+00,\n",
              "       3.00000000e+00, 1.00000000e+00, 2.40000000e+00, 7.07106781e-01,\n",
              "       1.59994000e+01, 5.58450000e+01, 3.98456000e+01, 3.19376400e+01,\n",
              "       2.81750940e+01, 6.00000000e-01, 1.40000000e+00, 8.00000000e-01,\n",
              "       9.20000000e-01, 5.65685425e-01, 6.10000000e+01, 1.01000000e+02,\n",
              "       4.00000000e+01, 8.50000000e+01, 2.82842712e+01, 0.00000000e+00,\n",
              "       0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,\n",
              "       3.17500000e+02, 4.91000000e+03, 4.59250000e+03, 2.15450000e+03,\n",
              "       3.24738789e+03, 2.65800000e-02, 8.00000000e+01, 7.99734200e+01,\n",
              "       3.20159480e+01, 5.65497476e+01, 5.48000000e+01, 1.81100000e+03,\n",
              "       1.75620000e+03, 7.57280000e+02, 1.24182093e+03, 0.00000000e+00,\n",
              "       0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,\n",
              "       0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,\n",
              "       0.00000000e+00])"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "comp = core.Composition(\"Fe2O3\")\n",
        "featurizer = dc.feat.ElementPropertyFingerprint()\n",
        "features = featurizer.featurize([comp])\n",
        "features[0]"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "f25f1177",
      "metadata": {
        "id": "f25f1177"
      },
      "source": [
        "## Datasets  <a class=\"anchor\" id=\"datasets\"></a>\n",
        "\n",
        "DeepChem has the following material properties dataset as part of MoleculeNet suite of datasets. These datasets can be used for a variety of tasks in material science like predicting structure formation energy, metallicity of a compound etc.\n",
        "\n",
        "- The [Band Gap dataset](https://deepchem.readthedocs.io/en/latest/moleculenet.html#deepchem.molnet.load_bandgap) contains 4604 experimentally measured band gaps for inorganic crystal structure compositions. The dataset can be loaded using `dc.molnet.load_bandgap` utility.\n",
        "- The [Perovskite dataset](https://deepchem.readthedocs.io/en/latest/moleculenet.html#deepchem.molnet.load_perovskite) contains 18928 perovskite structures and their formation energies. It can be loaded using a call to `dc.molnet.load_perovskite`.\n",
        "- The [Formation Energy dataset](https://deepchem.readthedocs.io/en/latest/moleculenet.html#deepchem.molnet.load_mp_formation_energy) contains 132752 calculated formation energies and inorganic crystal structures from the Materials Project database. It can be loaded using a call to `dc.molnet.load_mp_formation_energy`.\n",
        "- The [Metallicity dataset](https://deepchem.readthedocs.io/en/latest/moleculenet.html#deepchem.molnet.load_mp_metallicity) contains 106113 inorganic crystal structures from the Materials Project database labeled as metals or nonmetals. It can be loaded using `dc.molnet.load_mp_metallicity` utility.\n",
        "\n",
        "In the below example, we will demonstrate loading *perovskite* dataset and use it to predict *formation energy* of new crystals. Perovskite structures are structures adopted by many oxides. Ideally it is a cubic structure but non-cubic variants also exists. Each datapoint in the `perovskite` dataset contains the lattice structure as a `pymatgen.core.Structure` object and the formation energy of the corresponding structure. It can be used by calling for machine learning tasks by calling `dc.molnet.load_perovskite` utility. The utility takes care of loading, featurizing and splitting the dataset for machine learning tasks."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "id": "d6c18dd9",
      "metadata": {
        "id": "d6c18dd9"
      },
      "outputs": [],
      "source": [
        "dataset_config = {\"reload\": True, \"featurizer\": dc.feat.CGCNNFeaturizer(), \"transformers\": []}\n",
        "tasks, datasets, transformers = dc.molnet.load_perovskite(**dataset_config)\n",
        "train_dataset, valid_dataset, test_dataset = datasets"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "id": "129229ed",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 121
        },
        "id": "129229ed",
        "outputId": "b7e94a4c-3925-4227-fb66-254d4e472897"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div style=\"max-width:800px; border: 1px solid var(--colab-border-color);\"><style>\n",
              "      pre.function-repr-contents {\n",
              "        overflow-x: auto;\n",
              "        padding: 8px 12px;\n",
              "        max-height: 500px;\n",
              "      }\n",
              "\n",
              "      pre.function-repr-contents.function-repr-contents-collapsed {\n",
              "        cursor: pointer;\n",
              "        max-height: 100px;\n",
              "      }\n",
              "    </style>\n",
              "    <pre style=\"white-space: initial; background:\n",
              "         var(--colab-secondary-surface-color); padding: 8px 12px;\n",
              "         border-bottom: 1px solid var(--colab-border-color);\"><b>deepchem.data.datasets.DiskDataset.get_data_shape</b><br/>def get_data_shape() -&gt; Shape</pre><pre class=\"function-repr-contents function-repr-contents-collapsed\" style=\"\"><a class=\"filepath\" style=\"display:none\" href=\"#\">/usr/local/lib/python3.10/dist-packages/deepchem/data/datasets.py</a>Gets array shape of datapoints in this dataset.</pre>\n",
              "      <script>\n",
              "      if (google.colab.kernel.accessAllowed && google.colab.files && google.colab.files.view) {\n",
              "        for (const element of document.querySelectorAll('.filepath')) {\n",
              "          element.style.display = 'block'\n",
              "          element.onclick = (event) => {\n",
              "            event.preventDefault();\n",
              "            event.stopPropagation();\n",
              "            google.colab.files.view(element.textContent, 1570);\n",
              "          };\n",
              "        }\n",
              "      }\n",
              "      for (const element of document.querySelectorAll('.function-repr-contents')) {\n",
              "        element.onclick = (event) => {\n",
              "          event.preventDefault();\n",
              "          event.stopPropagation();\n",
              "          element.classList.toggle('function-repr-contents-collapsed');\n",
              "        };\n",
              "      }\n",
              "      </script>\n",
              "      </div>"
            ],
            "text/plain": [
              "<bound method DiskDataset.get_data_shape of <DiskDataset X.shape: (15142,), y.shape: (15142,), w.shape: (15142, 1), task_names: ['formation_energy']>>"
            ]
          },
          "execution_count": 9,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "train_dataset.get_data_shape"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "d36a9822",
      "metadata": {
        "id": "d36a9822"
      },
      "source": [
        "## Predicting Formation Energy  <a class=\"anchor\" id=\"pred-props\"></a>\n",
        "\n",
        "Along with the dataset and featurizers, DeepChem also provide implementation of various machine learning algorithms which can be used on the fly for material science applications. For predicting formation energy, we use `CGCNNModel` as described in the paper [1]."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "id": "4217a819",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 468
        },
        "id": "4217a819",
        "outputId": "c38976f8-9662-40cd-b1d7-7a22db032407"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Training: 100%|██████████| 10/10 [11:26<00:00, 68.65s/it]\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7967a9a126e0>]"
            ]
          },
          "execution_count": 10,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiwAAAGfCAYAAAB8wYmvAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA4XklEQVR4nO3de3jU9Z33/9fMJDOTMzmQCYFAOIRSDxAlJFKPq6nBtf2tW7sLbiuU3dted2td3bS1YFeoP6wBdb24KhRad61W60q9t4dtb5utjcZWG4GCeOYoh3DI5ERmkgmZJDNz/zGZCWMCZEKSOT0f1/W9knzznW8+c6U2Lz6f9/vzNfh8Pp8AAACimDHSAwAAALgQAgsAAIh6BBYAABD1CCwAACDqEVgAAEDUI7AAAICoR2ABAABRj8ACAACiHoEFAABEPQILAACIekmjedHmzZv12GOPqampSQsWLNCTTz6p8vLyYa/9xS9+oUceeUQHDx5UX1+fSkpK9M1vflN33nln8JqvfOUrevbZZ0NeV1VVpdra2hGNx+v16uTJk8rIyJDBYBjNWwIAABPM5/Ops7NThYWFMhovMIfiC9OLL77oM5vNvqefftr3wQcf+O666y7fpEmTfHa7fdjrX3vtNd8vfvEL34cffug7ePCgb+PGjT6TyeSrra0NXrNixQrfkiVLfKdOnQoe7e3tIx5TY2OjTxIHBwcHBwdHDB6NjY0X/Ftv8PnCe/hhRUWFFi1apE2bNknyz24UFRXpnnvu0apVq0Z0jyuvvFK33nqr1q1bJ8k/w9LR0aFf/epX4QwlyOFwaNKkSWpsbFRmZuao7gEAACaW0+lUUVGROjo6lJWVdd5rw1oS6u3t1a5du7R69ergOaPRqMrKSjU0NFzw9T6fT6+++qr27dunDRs2hHyvvr5e+fn5ys7O1o033qiHH35Yubm5w97H7XbL7XYHv+7s7JQkZWZmElgAAIgxIynnCCuwtLa2yuPxyGazhZy32Wzau3fvOV/ncDg0depUud1umUwm/fCHP9RnP/vZ4PeXLFmiL3zhC5o5c6YOHTqkBx54QLfccosaGhpkMpmG3K+mpkYPPfRQOEMHAAAxbFRFt+HKyMjQnj171NXVpbq6OlVXV2vWrFm64YYbJEnLli0LXnv55Zdr/vz5mj17turr63XTTTcNud/q1atVXV0d/DowpQQAAOJTWIElLy9PJpNJdrs95LzdbldBQcE5X2c0GjVnzhxJUmlpqT766CPV1NQEA8snzZo1S3l5eTp48OCwgcVischisYQzdAAAEMPC2ofFbDZr4cKFqqurC57zer2qq6vT4sWLR3wfr9cbUoPyScePH1dbW5umTJkSzvAAAECcCntJqLq6WitWrFBZWZnKy8u1ceNGuVwurVy5UpK0fPlyTZ06VTU1NZL89SZlZWWaPXu23G63Xn75ZT333HPasmWLJKmrq0sPPfSQbr/9dhUUFOjQoUO6//77NWfOHFVVVY3hWwUAALEq7MCydOlStbS0aM2aNWpqalJpaalqa2uDhbjHjh0L2fzF5XLp61//uo4fP66UlBTNmzdPzz//vJYuXSpJMplMevfdd/Xss8+qo6NDhYWFuvnmm7Vu3TqWfQAAgCQp7H1YopHT6VRWVpYcDgdtzQAAxIhw/n7zLCEAABD1CCwAACDqEVgAAEDUI7AAAICoR2ABAABRj8ByHo4zfdr4h/26//+8E+mhAACQ0Ags55FkNGjjHw7o5385rnZXb6SHAwBAwiKwnEeaJUnTslMkSQfsnREeDQAAiYvAcgFzbRmSpP3NXREeCQAAiYvAcgEltnRJzLAAABBJBJYLmJs/MMNCYAEAIGIILBcQWBI6YGdJCACASCGwXMCc/HQZDFKbq1dtXe5IDwcAgIREYLmAFLNJRdmpkqT9zLIAABARBJYRmBsovG2mjgUAgEggsIxAiY3CWwAAIonAMgIl+f4ZFpaEAACIDALLCAx2CnXK5/NFeDQAACQeAssIzJ7s7xQ63d2n1i6eKQQAwEQjsIxAitmk6Tn+TiF2vAUAYOIRWEaohB1vAQCIGALLCAVam3kIIgAAE4/AMkJnF94CAICJRWAZocBTm/fbu+gUAgBgghFYRmj25HQZDZLjTJ9aOnmmEAAAE4nAMkLWZJNm5KZJYgM5AAAmGoElDIM73lLHAgDARCKwhCFYeMtDEAEAmFAEljCcXXgLAAAmDoElDHPPemoznUIAAEwcAksYZk1Ok8loUGdPv+xOOoUAAJgoBJYwWJJMmpHrf6YQhbcAAEwcAkuY5vJMIQAAJhyBJUyBZwodoPAWAIAJQ2AJU0mg8JbWZgAAJgyBJUyBTqGDPFMIAIAJQ2AJ08y8NCUZDep09+uUoyfSwwEAICEQWMJkTjKqOC/wTCGWhQAAmAgEllGg8BYAgIlFYBmFElqbAQCYUASWUQhu0d/MDAsAABOBwDIKgSWhgzxTCACACTGqwLJ582YVFxfLarWqoqJCO3bsOOe1v/jFL1RWVqZJkyYpLS1NpaWleu6550Ku8fl8WrNmjaZMmaKUlBRVVlbqwIEDoxnahCjOS1OyySBXr0cnOs5EejgAAMS9sAPLtm3bVF1drbVr12r37t1asGCBqqqq1NzcPOz1OTk5+u53v6uGhga9++67WrlypVauXKn/+Z//CV7z6KOP6gc/+IG2bt2q7du3Ky0tTVVVVerpic624WSTUTMHOoUovAUAYPwZfGGuaVRUVGjRokXatGmTJMnr9aqoqEj33HOPVq1aNaJ7XHnllbr11lu1bt06+Xw+FRYW6pvf/Ka+9a1vSZIcDodsNpueeeYZLVu27IL3czqdysrKksPhUGZmZjhvZ9TufmG3/u+7p/TAX8/TV6+bPSE/EwCAeBLO3++wZlh6e3u1a9cuVVZWDt7AaFRlZaUaGhou+Hqfz6e6ujrt27dP1113nSTp8OHDampqCrlnVlaWKioqznlPt9stp9MZcky0wYcgMsMCAMB4CyuwtLa2yuPxyGazhZy32Wxqamo65+scDofS09NlNpt166236sknn9RnP/tZSQq+Lpx71tTUKCsrK3gUFRWF8zbGxOBeLLQ2AwAw3iakSygjI0N79uzRzp079f3vf1/V1dWqr68f9f1Wr14th8MRPBobG8dusCMUeAjigeYueb10CgEAMJ6Swrk4Ly9PJpNJdrs95LzdbldBQcE5X2c0GjVnzhxJUmlpqT766CPV1NTohhtuCL7ObrdrypQpIfcsLS0d9n4Wi0UWiyWcoY+54txUJZsM6h7oFCrKSY3oeAAAiGdhzbCYzWYtXLhQdXV1wXNer1d1dXVavHjxiO/j9XrldrslSTNnzlRBQUHIPZ1Op7Zv3x7WPSdaksmoWXkDy0LNLAsBADCewpphkaTq6mqtWLFCZWVlKi8v18aNG+VyubRy5UpJ0vLlyzV16lTV1NRI8teblJWVafbs2XK73Xr55Zf13HPPacuWLZIkg8Gg++67Tw8//LBKSko0c+ZMPfjggyosLNRtt902du90HJTY0rXP3qn99i7dOM924RcAAIBRCTuwLF26VC0tLVqzZo2amppUWlqq2traYNHssWPHZDQOTty4XC59/etf1/Hjx5WSkqJ58+bp+eef19KlS4PX3H///XK5XPrqV7+qjo4OXXPNNaqtrZXVah2Dtzh+/Fv0n+KZQgAAjLOw92GJRpHYh0WSat8/pf/9/G5dPjVLv7nnmgn7uQAAxINx24cFoQKdQgfpFAIAYFwRWC7CjJxUmU1Gnenz6PhpnikEAMB4IbBchCSTUbMm+58pRB0LAADjh8BykeYOLAvtp7UZAIBxQ2C5SINb9PNMIQAAxguB5SIFCm9ZEgIAYPwQWC7S3LM6hTx0CgEAMC4ILBdpek6qLElGufu9amzvjvRwAACISwSWi2QyGjR7sr+OhWUhAADGB4FlDAQLb5spvAUAYDwQWMYAhbcAAIwvAssYCO7FQmszAADjgsAyBgJLQoda6BQCAGA8EFjGQFF2qqzJRvX2e3W0zRXp4QAAEHcILGPAaDRoTn6gU4hlIQAAxhqBZYzMzffXsRyg8BYAgDFHYBkjwU4hWpsBABhzBJYxMvgQRGZYAAAYawSWMRJobf64xaV+jzfCowEAIL4QWMbI1EkpSkk2qdfj1ZE2nikEAMBYIrCMEaPRoBKWhQAAGBcEljFUks+OtwAAjAcCyxgKFN7ub2aGBQCAsURgGUOBwluWhAAAGFsEljEUqGE53OpSH51CAACMGQLLGJo6KUVpZpP6PD4daeWZQgAAjBUCyxgyGAyaY6PwFgCAsUZgGWNzBx6CeIDCWwAAxgyBZYwNFt4ywwIAwFghsIyxOYHWZjqFAAAYMwSWMRaYYTnc6lJvP51CAACMBQLLGCvMsirdkqR+r09H2ugUAgBgLBBYxpjBYNCcfJaFAAAYSwSWcRDcop/CWwAAxgSBZRywRT8AAGOLwDIOSoKbxxFYAAAYCwSWcRBYEjrS1i13vyfCowEAIPYRWMZBQaZVGZYkebw+HeaZQgAAXDQCyzgwGAzBJzdTeAsAwMUjsIwTCm8BABg7BJZxQuEtAABjZ1SBZfPmzSouLpbValVFRYV27NhxzmufeuopXXvttcrOzlZ2drYqKyuHXP+Vr3xFBoMh5FiyZMlohhY1AoW3PAQRAICLF3Zg2bZtm6qrq7V27Vrt3r1bCxYsUFVVlZqbm4e9vr6+XnfccYdee+01NTQ0qKioSDfffLNOnDgRct2SJUt06tSp4PGf//mfo3tHUSKwJHSkzaWePjqFAAC4GGEHlieeeEJ33XWXVq5cqUsuuURbt25Vamqqnn766WGv/9nPfqavf/3rKi0t1bx58/Tv//7v8nq9qqurC7nOYrGooKAgeGRnZ4/uHUWJ/AyLMq1J8vqkj1voFAIA4GKEFVh6e3u1a9cuVVZWDt7AaFRlZaUaGhpGdI/u7m719fUpJycn5Hx9fb3y8/P1qU99Sl/72tfU1tZ2znu43W45nc6QI9oYDIbBwttm6lgAALgYYQWW1tZWeTwe2Wy2kPM2m01NTU0jusd3vvMdFRYWhoSeJUuW6Kc//anq6uq0YcMGvf7667rlllvk8Qy/lFJTU6OsrKzgUVRUFM7bmDAU3gIAMDaSJvKHrV+/Xi+++KLq6+tltVqD55ctWxb8/PLLL9f8+fM1e/Zs1dfX66abbhpyn9WrV6u6ujr4tdPpjMrQwkMQAQAYG2HNsOTl5clkMslut4ect9vtKigoOO9rH3/8ca1fv16///3vNX/+/PNeO2vWLOXl5engwYPDft9isSgzMzPkiEbsxQIAwNgIK7CYzWYtXLgwpGA2UEC7ePHic77u0Ucf1bp161RbW6uysrIL/pzjx4+rra1NU6ZMCWd4USew2+3R9m46hQAAuAhhdwlVV1frqaee0rPPPquPPvpIX/va1+RyubRy5UpJ0vLly7V69erg9Rs2bNCDDz6op59+WsXFxWpqalJTU5O6uvzLJF1dXfr2t7+tt956S0eOHFFdXZ3+5m/+RnPmzFFVVdUYvc3ImJxu0aTUZPl80sFmloUAABitsGtYli5dqpaWFq1Zs0ZNTU0qLS1VbW1tsBD32LFjMhoHc9CWLVvU29urL37xiyH3Wbt2rb73ve/JZDLp3Xff1bPPPquOjg4VFhbq5ptv1rp162SxWC7y7UWWwWDQ3PwM7TjSrgPNnbpsalakhwQAQEwy+Hw+X6QHcbGcTqeysrLkcDiirp7lu798Tz/bfkxfu2G2vrNkXqSHAwBA1Ajn7zfPEhpnFN4CAHDxCCzjrITWZgAALhqBZZwFZlgaT3frTC+dQgAAjAaBZZzlpVuUk2amUwgAgItAYJkAJfmBZSHqWAAAGA0CywQILAvt5yGIAACMCoFlAgSeKXSAwlsAAEaFwDIBeGozAAAXh8AyAQJLQsdPn5HL3R/h0QAAEHsILBMgJ82svHSzJDqFAAAYDQLLBCnJZ1kIAIDRIrBMkMCOt8ywAAAQPgLLBKHwFgCA0SOwTJC5+TxTCACA0SKwTJBAp9CJDjqFAAAIF4FlgmSnmZWXbpEkHaCOBQCAsBBYJlBgx1vqWAAACA+BZQIFloUOEFgAAAgLgWUCldgovAUAYDQILBOIGRYAAEaHwDKB5g7sdnvS0aPOnr4IjwYAgNhBYJlAWanJys+gUwgAgHARWCYYy0IAAISPwDLBKLwFACB8BJYJNpdnCgEAEDYCywQLbB53gBkWAABGjMAyweYMdAo1OXvkOEOnEAAAI0FgmWBZKckqyLRKkg42sywEAMBIEFgigMJbAADCQ2CJAApvAQAID4ElAii8BQAgPASWCChhhgUAgLAQWCKgJN8/w9Lc6Zajm04hAAAuhMASARnWZBVm+TuF9tMpBADABRFYIoRlIQAARo7AEiEU3gIAMHIElghhhgUAgJEjsETI4F4szLAAAHAhBJYICXQKtXa5ddrVG+HRAAAQ3QgsEZJmSdLUSSmSWBYCAOBCCCwRFCi83d/MshAAAOdDYImgQB3LAWZYAAA4r1EFls2bN6u4uFhWq1UVFRXasWPHOa996qmndO211yo7O1vZ2dmqrKwccr3P59OaNWs0ZcoUpaSkqLKyUgcOHBjN0GIKnUIAAIxM2IFl27Ztqq6u1tq1a7V7924tWLBAVVVVam5uHvb6+vp63XHHHXrttdfU0NCgoqIi3XzzzTpx4kTwmkcffVQ/+MEPtHXrVm3fvl1paWmqqqpST0/P6N9ZDGAvFgAARsbg8/l84bygoqJCixYt0qZNmyRJXq9XRUVFuueee7Rq1aoLvt7j8Sg7O1ubNm3S8uXL5fP5VFhYqG9+85v61re+JUlyOByy2Wx65plntGzZsgve0+l0KisrSw6HQ5mZmeG8nYhyuft16dr/kSTt+tdK5aZbIjwiAAAmTjh/v8OaYent7dWuXbtUWVk5eAOjUZWVlWpoaBjRPbq7u9XX16ecnBxJ0uHDh9XU1BRyz6ysLFVUVJzznm63W06nM+SIRWmWJE3LDnQKMcsCAMC5hBVYWltb5fF4ZLPZQs7bbDY1NTWN6B7f+c53VFhYGAwogdeFc8+amhplZWUFj6KionDeRlQJFt7yEEQAAM5pQruE1q9frxdffFG//OUvZbVaR32f1atXy+FwBI/GxsYxHOXEKqGOBQCAC0oK5+K8vDyZTCbZ7faQ83a7XQUFBed97eOPP67169frD3/4g+bPnx88H3id3W7XlClTQu5ZWlo67L0sFosslvio95ibT6cQAAAXEtYMi9ls1sKFC1VXVxc85/V6VVdXp8WLF5/zdY8++qjWrVun2tpalZWVhXxv5syZKigoCLmn0+nU9u3bz3vPeDG4JMQMCwAA5xLWDIskVVdXa8WKFSorK1N5ebk2btwol8ullStXSpKWL1+uqVOnqqamRpK0YcMGrVmzRi+88IKKi4uDdSnp6elKT0+XwWDQfffdp4cfflglJSWaOXOmHnzwQRUWFuq2224bu3capebkp8tgkNpdvWrtciuPTiEAAIYIO7AsXbpULS0tWrNmjZqamlRaWqra2tpg0eyxY8dkNA5O3GzZskW9vb364he/GHKftWvX6nvf+54k6f7775fL5dJXv/pVdXR06JprrlFtbe1F1bnEihSzSUXZqTrW3q399k4CCwAAwwh7H5ZoFKv7sAT8r2d36g8fNeuh/+9SrfhMcaSHAwDAhBi3fVgwPtiiHwCA8yOwRAG26AcA4PwILFGgJNDa3NypOFihAwBgzBFYosCc/HQZDVJHd59autyRHg4AAFGHwBIFrMkmTc9JlcSyEAAAwyGwRAkKbwEAODcCS5QIFN7y1GYAAIYisESJ4Bb9zLAAADAEgSVKlJz1EEQ6hQAACEVgiRKzJqfJaJCcPf1q7qRTCACAsxFYooQ12aTi3DRJFN4CAPBJBJYoUkLhLQAAwyKwRBEKbwEAGB6BJYqwFwsAAMMjsESRsx+CSKcQAACDCCxRZGZemkxGgzrd/Wpy9kR6OAAARA0CSxSxJJlUnOt/phCFtwAADCKwRBkKbwEAGIrAEmUovAUAYCgCS5ThIYgAAAxFYIkygSWhg810CgEAEEBgiTLFuWlKMhrU5e7XSQedQgAASASWqGNOMmpmHs8UAgDgbASWKESnEAAAoQgsUWhOPoW3AACcjcAShZhhAQAgFIElCgWfKdTcJa+XTiEAAAgsUag4L03JJoO6ez060XEm0sMBACDiCCxRKNk02Cl0oJllIQAACCxRanCLfgpvAQAgsESpufmBwlsCCwAABJYoNVh4y5IQAAAElihVYhucYaFTCACQ6AgsUao4N1Vmk1Fn+ugUAgCAwBKlkkxGzZrMM4UAAJAILFGNTiEAAPwILFFs7sAzhdiiHwCQ6AgsUSw4w0KnEAAgwRFYoligtfkgzxQCACQ4AksUm5GbJnOSUT19XjWe7o70cAAAiBgCSxQzGQ2aPdk/y0LhLQAgkRFYolxgWYjWZgBAIhtVYNm8ebOKi4tltVpVUVGhHTt2nPPaDz74QLfffruKi4tlMBi0cePGIdd873vfk8FgCDnmzZs3mqHFnbnBHW8JLACAxBV2YNm2bZuqq6u1du1a7d69WwsWLFBVVZWam5uHvb67u1uzZs3S+vXrVVBQcM77XnrppTp16lTweOONN8IdWlwqyWdJCACAsAPLE088obvuuksrV67UJZdcoq1btyo1NVVPP/30sNcvWrRIjz32mJYtWyaLxXLO+yYlJamgoCB45OXlnfNat9stp9MZcsSrwAzLoZYueegUAgAkqLACS29vr3bt2qXKysrBGxiNqqysVENDw0UN5MCBAyosLNSsWbP0pS99SceOHTvntTU1NcrKygoeRUVFF/Wzo1lRTqosSUa5+7061k6nEAAgMYUVWFpbW+XxeGSz2ULO22w2NTU1jXoQFRUVeuaZZ1RbW6stW7bo8OHDuvbaa9XZOXzdxurVq+VwOIJHY2PjqH92tDMZDZqTT+EtACCxJUV6AJJ0yy23BD+fP3++KioqNGPGDP385z/XP/3TPw253mKxnHd5Kd7MtWXog5NOHbB3qurSc9cBAQAQr8KaYcnLy5PJZJLdbg85b7fbz1tQG65JkyZp7ty5Onjw4JjdM5aV2Ci8BQAktrACi9ls1sKFC1VXVxc85/V6VVdXp8WLF4/ZoLq6unTo0CFNmTJlzO4Zy+bmB57azJIQACAxhb0kVF1drRUrVqisrEzl5eXauHGjXC6XVq5cKUlavny5pk6dqpqaGkn+Qt0PP/ww+PmJEye0Z88epaena86cOZKkb33rW/r85z+vGTNm6OTJk1q7dq1MJpPuuOOOsXqfMS3QKfRxi0v9Hq+STOz3BwBILGEHlqVLl6qlpUVr1qxRU1OTSktLVVtbGyzEPXbsmIzGwT+oJ0+e1BVXXBH8+vHHH9fjjz+u66+/XvX19ZKk48eP64477lBbW5smT56sa665Rm+99ZYmT558kW8vPkzLTlFKskln+jw62t4d3K4fAIBEYfD5fDG/uYfT6VRWVpYcDocyMzMjPZxx8fkn39B7Jxza+uUrteQylsoAALEvnL/frC3ECApvAQCJjMASIwJ1LBTeAgASEYElRgSe2nyAGRYAQAIisMSIkoHW5o9bu9Tn8UZ4NAAATCwCS4yYOsnfKdTn8elomyvSwwEAYEIRWGKE0Wig8BYAkLAILDGkhB1vAQAJisASQyi8BQAkKgJLDKG1GQCQqAgsMSRQw3K41aXefjqFAACJg8ASQ6ZOSlGa2aR+r09H6BQCACQQAksMMRgMmsOyEAAgARFYYszcfFqbAQCJh8ASYwKFtwebmWEBACQOAkuMYfM4AEAiIrDEmMAMyxE6hQAACYTAEmOmZFmVYUlSv9enw610CgEAEgOBJcb4O4UCy0LUsQAAEgOBJQbNHXim0AECCwAgQRBYYhCFtwCARENgiUHBZwrR2gwASBAElhgUCCxH27rl7vdEeDQAAIw/AksMsmValGFNksfr08ctdAoBAOIfgSUGGQyGwWUhCm8BAAmAwBKj5g4U3h6g8BYAkAAILDGqJJ8ZFgBA4iCwxKjAktCBZmZYAADxj8ASowJLQkfbXOrpo1MIABDfCCwxanKGRVkpyfL6pEMtzLIAAOIbgSVG+TuFKLwFACQGAksMK6G1GQCQIAgsMWxuPs8UAgAkBgJLDBvsFGKGBQAQ3wgsMSywJHSsvVtneukUAgDELwJLDMtLNys7NVk+OoUAAHGOwBLDDAYDhbcAgIRAYIlxgdZmCm8BAPGMwBLjgoW3zLAAAOIYgSXGzQm0NtMpBACIYwSWGBeYYWlsP6Pu3v4IjwYAgPFBYIlxeekW5aSZJUkHeXIzACBOjSqwbN68WcXFxbJaraqoqNCOHTvOee0HH3yg22+/XcXFxTIYDNq4ceNF3xOhStjxFgAQ58IOLNu2bVN1dbXWrl2r3bt3a8GCBaqqqlJzc/Ow13d3d2vWrFlav369CgoKxuSeCEXhLQAg3oUdWJ544gndddddWrlypS655BJt3bpVqampevrpp4e9ftGiRXrssce0bNkyWSyWMbknQg22NhNYAADxKazA0tvbq127dqmysnLwBkajKisr1dDQMKoBjOaebrdbTqcz5Ehkg5vHsSQEAIhPYQWW1tZWeTwe2Wy2kPM2m01NTU2jGsBo7llTU6OsrKzgUVRUNKqfHS8CS0InOs7I5aZTCAAQf2KyS2j16tVyOBzBo7GxMdJDiqicNLPy0v2dQgfoFAIAxKGkcC7Oy8uTyWSS3W4POW+3289ZUDse97RYLOesh0lUJfkZau1q0357p0qLJkV6OAAAjKmwZljMZrMWLlyourq64Dmv16u6ujotXrx4VAMYj3smokDhLZ1CAIB4FNYMiyRVV1drxYoVKisrU3l5uTZu3CiXy6WVK1dKkpYvX66pU6eqpqZGkr+o9sMPPwx+fuLECe3Zs0fp6emaM2fOiO6JCwsU3rIkBACIR2EHlqVLl6qlpUVr1qxRU1OTSktLVVtbGyyaPXbsmIzGwYmbkydP6oorrgh+/fjjj+vxxx/X9ddfr/r6+hHdExc2uBcLgQUAEH8MPp/PF+lBXCyn06msrCw5HA5lZmZGejgR0dHdq9L//xVJ0vsPVSndEnYWBQBgQoXz9zsmu4Qw1KRUsyZn+AuRqWMBAMQbAkscGSy8ZVkIABBfCCxxpCQ/sOMtMywAgPhCYIkjgcLb/XQKAQDiDIEljrAXCwAgXhFY4khgL5ZTjh45e/oiPBoAAMYOgSWOZKUky5YZ6BRiWQgAED8ILHFmcAM5loUAAPGDwBJnBjuFmGEBAMQPAkucCRbeNjPDAgCIHwSWOBMovGUvFgBAPCGwxJmSgRkWu9Mtxxk6hQAA8YHAEmcyrcmakmWVROEtACB+EFji0OCyEIW3AID4QGCJQ3Pz/ctC1LEAAOIFgSUOBfdioVMIABAnCCxxKFB4y5IQACBeEFjiUKCGpaXTrY7u3giPBgCAi0dgiUPpliRNnZQiiVkWAEB8ILDEqTkU3gIA4giBJU4Ft+gnsAAA4gCBJU6xFwsAIJ4QWOIUrc0AgHhCYIlTJQM1LK1dvWp30SkEAIhtBJY4lRbSKcQsCwAgthFY4hiFtwCAeEFgiWNzKbwFAMQJAkscG+wUYoYFABDbCCxxLLgk1MwMCwAgthFY4lhgt9t2V69au9wRHg0AAKNHYIljqeYkFeXQKQQAiH0Eljg3N39gAzkKbwEAMYzAEucovAUAxAMCS5yj8BYAEA8ILHEu+Ewhe6d8Pl+ERwMAwOgQWOLc7MnpMhik0919au3imUIAgNhEYIlzKWaTpuekSmKLfgBA7CKwJICSfApvAQCxjcCSAAKFt/spvAUAxCgCSwI4u/AWAIBYRGBJACWBGRZ7F51CAICYRGBJALMnp8tokBxn+rT86R36yZuHdbTNFelhAQAwYqMKLJs3b1ZxcbGsVqsqKiq0Y8eO817/0ksvad68ebJarbr88sv18ssvh3z/K1/5igwGQ8ixZMmS0QwNw7Amm3TLZVMkSX860KqHfvOhrn+sXjf+W70e/u2H+vPBVvX2eyM8SgAAzs3gC3ONYNu2bVq+fLm2bt2qiooKbdy4US+99JL27dun/Pz8Idf/+c9/1nXXXaeamhp97nOf0wsvvKANGzZo9+7duuyyyyT5A4vdbtdPfvKT4OssFouys7NHNCan06msrCw5HA5lZmaG83YShs/n06GWLr26t1mv7m3WX46cVr938FefbknSdXPz9FefytcNn8rX5AxLBEcLAEgE4fz9DjuwVFRUaNGiRdq0aZMkyev1qqioSPfcc49WrVo15PqlS5fK5XLpt7/9bfDcVVddpdLSUm3dulWSP7B0dHToV7/6VThDCSKwhM/Z06c/7W/Vq3ubVb+vWW2u0E3lFkzL0l/Ny9eN8/J1WWGWjEZDhEYKAIhX4fz9Tgrnxr29vdq1a5dWr14dPGc0GlVZWamGhoZhX9PQ0KDq6uqQc1VVVUPCSX19vfLz85Wdna0bb7xRDz/8sHJzc4e9p9vtltvtDn7tdDrDeRuQlGlN1q3zp+jW+VPk9fr0zvEOvba3Wa/ua9b7J5x657hD7xx3aOMfDmhyhkU3zJ2sG+fl65qSPGVYkyM9fABAggkrsLS2tsrj8chms4Wct9ls2rt377CvaWpqGvb6pqam4NdLlizRF77wBc2cOVOHDh3SAw88oFtuuUUNDQ0ymUxD7llTU6OHHnoonKHjPIxGg66Ynq0rpmer+uZPye7sUf0+/9LRGwda1dLp1ku7juulXceVbDJoUXGObhyYfZk1OT3SwwcAJICwAst4WbZsWfDzyy+/XPPnz9fs2bNVX1+vm266acj1q1evDpm1cTqdKioqmpCxJgJbplVLF03X0kXT5e73aOfh0wO1L3YdaevWnw+16c+H2vTw//1IxbmpwaWj8pk5siQNDZgAxk9vv1fvnejQ9sPt2nm4XXubOlVaNEl3Lp6hxbNyZTCwnIv4EFZgycvLk8lkkt1uDzlvt9tVUFAw7GsKCgrCul6SZs2apby8PB08eHDYwGKxWGSxUBQ6ESxJJl1TkqdrSvK05vOX6OOBwt3X9jVrx+F2HWnr1k/ePKKfvHlEaWaTrp6Tpxvn5euv5uXLlmmN9PCBuNPd26+3j/kDyo7DbXr7WIfcn+jyO+Vo0u/eb1JJfrruXDxDf3vFVJZyEfPCCixms1kLFy5UXV2dbrvtNkn+otu6ujp94xvfGPY1ixcvVl1dne67777guVdeeUWLFy8+5885fvy42traNGXKlHCGhwkwa3K6Zk1O1/+6dpY6e/r05sHWgQDTopZOt37/oV2//9AfUC8tzNRNA+FlwbRJFO4Co+Do7tNfjrZrx+F2bT/crvdPOEI6/CQpJ82s8uIcLZqZo7m2dNW+36Rfvn1CB5q7tObXH2jD7/bqb6+cquWLi4M7XwOxZlRtzStWrNCPfvQjlZeXa+PGjfr5z3+uvXv3ymazafny5Zo6dapqamok+duar7/+eq1fv1633nqrXnzxRT3yyCPBtuauri499NBDuv3221VQUKBDhw7p/vvvV2dnp957770RzaTQJRR5Xq9PH5x0BpeO3jnuCPl+bppZ13/KX7h7bclkZaXwrz1gOM2dPdp5+LR2HG7T9sPt2mfv1Cf/X7owy6rymTkqn5mr8pnZmj05fcjSj7OnT7/YdVzPvXVUh1oGN4qsmJmjOxfPUNWlBUo2sXcoImtc25oladOmTXrsscfU1NSk0tJS/eAHP1BFRYUk6YYbblBxcbGeeeaZ4PUvvfSS/vVf/1VHjhxRSUmJHn30Uf31X/+1JOnMmTO67bbb9Pbbb6ujo0OFhYW6+eabtW7duiHFumPxhjExWjrdqt/nXzr60/5Wdbr7g98zGQ0qm5EdLNydkz/0/2yBRODz+XT89Jlg/cmOI+063Dp0F+pZk9NUXpwzEFJyNC07Nayf0XCoTT9tOKpXPrLLMzA7k59h0R3l0/UPFdNZvkXEjHtgiTYElujW5/Fq55F2f9v03uaQf+1J0rTslODS0VWzcmVNpnAX8cnn8+lgc9dA/Um7dh5p1ylHT8g1BoP06YLMYDhZVJwzZhs5nnKc0Qvbj+k/dzSqtcu/NUSS0aCqSwt05+IZqpiZwz8eMKEILIhqR9tcwR13t3/crl7PYMFgSrJJV8/JDXYeTclKieBIgYvT7/Hqo1Od2n64TTsOt+svR0+r/RObNCYZDZo/LSu4vLNwRs64L5n29ntV+0GTnms4op1HTgfPz7Wl686rZuhvr5ymdEtUNJEizhFYEDNc7n69ebBVrw3s+2J3ukO+P68gI7h0dMX0bJko3EUUc/d79O5xR7BAdteRdrl6PSHXWJONunJ6dnAG5YqibKWYIzer+OFJp57fflS/3H1CZ/r8Y023JOkLV07VnVfNUAlFuhhHBBbEJJ/Ppw9POYNLR283doQUG05KTdb1cyerYmau0q1JsiQZZUkyyppsGvjcJGuyUZbg1/7vJRkNTHNjXHS5+7X76GntPOIPKHsaO4Y8SDTDmqRFZ9WfXFaYJXNS9BW7Onv69F8DRbofn7Vse9WsHC1fXKzPXmKjSBdjjsCCuNDW5dbr+1v06t5m/XF/i5w9/Rd+0TCMBg2GmSSTLMlGWQc+fjLwWALXnH0+5Pqz7vOJa4b7HjNC8eW0q1c7j/hrT3Ycbtf7J53BItaAvHSLymdmDxTJ5upTBRkx9b8Dn8+nNw+26acNR/SHj+wKvD1bpkX/UD5Dd5QXKZ8iXYwRAgviTr/Hq11HT+vVfc3a39Spnj6v3P0eufu96unzfzz780/+KzdSkoyGTwSbwY+pySblZVg0Od2iyRlnHekW5WWYlZtmiak/dPGoydGjHUf8G7TtPHxa++ydQ66Zlp3inz0ZmEWZmZcWNzN6Jzv8Rbov7jym1i5/7U2S0aCqywq0/KoZKqdIFxeJwIKE5/X61Ovxyj0QbIYPOGed7xs++AS/1++Vu29oQHIPc68+z9j8J2U0SDlpoUHmk8FmcoZZk9OtykxJ4g/HRfL5fDrW3h3SwXO0rXvIdXPy04MBZdHMHE2dFP+F4e5+j2rfb9JPG45q19HBIt15BRn68lX+nXTTKNLFKBBYgAjyeH3BoHO+gOTq7VdLp1stXW7/x4GjtcutNlfvkM3CzsdsMmpyhuUcMzbmgY9WTc6wRLTAczz5fD719HnV6e5TV0+/utz96uzxH13ufnX19AU/73T3q6unX509fcHrWrvcwVmEAKNBuqQwU+XFuQMtxtnKTU/sx4J8cNKh5986ql+9fTKkSPf2K6fqzsUzNCefIl2MHIEFiHH9Hq/aXb2hYeYTwSbwdWeYtT3plqQhMzZ5gVBzVrDJTTdPSJGlz+eTu987GCZ6/IFjSKgYCBZdwQDSL+fA9wJff3LL+nCZTUYtKMoKFskunJHNM3jOwXGmT/9n13E9/9bRkM3uPjM7V8sXz1Dlp21KokgXF0BgARJIT59HrRcINq1dbjU73UMeknchOWnmYLAJCTVnBZtJqcnq7vUMhIy+0FDhHggVZ4WMzkAYGZgJ6RyDoHE2g8EfyjIsSUq3JinDmqz0wOeWJGVYk5RuSQ792pqkTGuyPlWQwcaFYfJ6fXrzUKt+2nBUdWcV6RZkWvUPFdO1rLxI+RkU6WJ4BBYAQ/h8PnW5+88ZbFq7Bs+1dvUO6X4ZbwaDlG72h4f0YJBI9gePs4JFusUfLgKf+8PGYAhJTTbxoM0IOX66Wy9sP6ZtOxvVNrBBXrLJoCWXTdHyxTNUNiObWiuEILAAuCher0+nu0OXpM41i9Nxpk9p5qGhIsOapAxLcujXZwULf/AYvD7NnETQiBPufo9+916TftpwRLuPdQTPzyvI0J2LZ+i2Uop04UdgATBhfD4f/2rGOb1/wqHnGo7q1++cUE+ff0kyw5Kk2xdO052LZ2j25PQIjxCRRGABAEQVR3efXtrVqOffOqojZ7WLXz0nV3deVazKT+dTpJuACCwAgKjk9fr0p4Oteq7hqF7dO1ikW5jlL9Jdumj6mD2dGtGPwAIAiHqN7d16YYe/SLf9rCLdWwaKdK+cnk1dU5wjsAAAYkZPn0cvv3dKP204qj2NHcHzRoM0KdWsSanJyk41DxzJyk4b/HxSqlk5aYOfT0pN5iGNMYTAAgCISe8dd+i5t47ov985GSzSDVeGNWlIuJmUmqycVLMmDYSbnFSzJqWalZ3mD0PsvxMZBBYAQEzr83h1urtXp119Ot3dq47uXrV/4vOO7l7/Nd3+844zfWE90uJsKcmmoQEnbSDUBGZ40kI/TzOb6JC7SOH8/aYRHgAQdZJNRuVnWMPaJdfj9clxZgQBZ+D86W7/uX6vT2f6PDrj8OikoyeMMRr8S1JnL1udFWoCoacgy6rpOak85uEiEVgAAHHBZDQoJ81f0zJSPp9Pne5+dbj61D4QaM4XcE67/Ofc/V71eXzBDRRHIjs1WdNz0zQ9J1XTc1IGPqZpem6qCjKtMlFgfF4EFgBAwjIYDMq0JivTmqzpuakjft2ZXo8/4Lh61dHtDzsdIeHGH3DaXW6d6uhRmyuwdNWhd84qLA5INhk0LTt1IMT4j6KcVM3I9X9MZ2dgAgsAAOFKMZs01ZyiqZNSRnR9Z0+fGtvP6Fh7txrbu3W03aVj7WfU2N6t46e71efx6XCrK+TJ12fLTTOraCDIBEJMINgUZFoTov2bwAIAwDjLsCbrksJkXVI4tLDU4/XplGMwzBxr79bRtsHPT3f3qc3VqzZXb0jbd4DZZNS04BLTWUduqoqyU+PmuU3x8S4AAIhRJqN/OWhadqo0e+j3nT19OnZWgDn7OHH6jHo9Xn3c4tLHLcPPzuSlW86qmUk9q44mVfkZlpiZnaGtGQCAGNXv8eqUo2dIkDnW5v/oONN33tdbkowhy0shy07ZqUoxj+/+NLQ1AwCQAJJM/sBRlJOqq4f5vqO7T42n/UtMgTATqKE52dEjd79XB5u7dLC5a9j7T86w+APMwM/42g2zI7bJHoEFAIA4lZWarKzULF02NWvI9/o8Xp3qGJydOdruCqmh6ezpD7Zt7zp6WuYko+69qSQC78KPwAIAQAJKNhk1PTf1nO3cHd29IctM3W5PROtdCCwAAGCISQPPW5o/bVKkhyJJ4pGWAAAg6hFYAABA1COwAACAqEdgAQAAUY/AAgAAoh6BBQAARD0CCwAAiHoEFgAAEPUILAAAIOoRWAAAQNQjsAAAgKhHYAEAAFGPwAIAAKJeXDyt2efzSZKcTmeERwIAAEYq8Hc78Hf8fOIisHR2dkqSioqKIjwSAAAQrs7OTmVlZZ33GoNvJLEmynm9Xp08eVIZGRkyGAxjem+n06mioiI1NjYqMzNzTO+N8PH7iC78PqIPv5Powu/j/Hw+nzo7O1VYWCij8fxVKnExw2I0GjVt2rRx/RmZmZn8jy2K8PuILvw+og+/k+jC7+PcLjSzEkDRLQAAiHoEFgAAEPUILBdgsVi0du1aWSyWSA8F4vcRbfh9RB9+J9GF38fYiYuiWwAAEN+YYQEAAFGPwAIAAKIegQUAAEQ9AgsAAIh6BBYAABD1CCwXsHnzZhUXF8tqtaqiokI7duyI9JASUk1NjRYtWqSMjAzl5+frtttu0759+yI9LAxYv369DAaD7rvvvkgPJWGdOHFCX/7yl5Wbm6uUlBRdfvnl+stf/hLpYSUkj8ejBx98UDNnzlRKSopmz56tdevWjegBfzg3Ast5bNu2TdXV1Vq7dq12796tBQsWqKqqSs3NzZEeWsJ5/fXXdffdd+utt97SK6+8or6+Pt18881yuVyRHlrC27lzp370ox9p/vz5kR5Kwjp9+rSuvvpqJScn63e/+50+/PBD/du//Zuys7MjPbSEtGHDBm3ZskWbNm3SRx99pA0bNujRRx/Vk08+GemhxTT2YTmPiooKLVq0SJs2bZLkf8hiUVGR7rnnHq1atSrCo0tsLS0tys/P1+uvv67rrrsu0sNJWF1dXbryyiv1wx/+UA8//LBKS0u1cePGSA8r4axatUpvvvmm/vSnP0V6KJD0uc99TjabTf/xH/8RPHf77bcrJSVFzz//fARHFtuYYTmH3t5e7dq1S5WVlcFzRqNRlZWVamhoiODIIEkOh0OSlJOTE+GRJLa7775bt956a8h/J5h4//3f/62ysjL93d/9nfLz83XFFVfoqaeeivSwEtZnPvMZ1dXVaf/+/ZKkd955R2+88YZuueWWCI8stsXF05rHQ2trqzwej2w2W8h5m82mvXv3RmhUkPwzXffdd5+uvvpqXXbZZZEeTsJ68cUXtXv3bu3cuTPSQ0l4H3/8sbZs2aLq6mo98MAD2rlzp/75n/9ZZrNZK1asiPTwEs6qVavkdDo1b948mUwmeTweff/739eXvvSlSA8tphFYEHPuvvtuvf/++3rjjTciPZSE1djYqHvvvVevvPKKrFZrpIeT8Lxer8rKyvTII49Ikq644gq9//772rp1K4ElAn7+85/rZz/7mV544QVdeuml2rNnj+677z4VFhby+7gIBJZzyMvLk8lkkt1uDzlvt9tVUFAQoVHhG9/4hn7729/qj3/8o6ZNmxbp4SSsXbt2qbm5WVdeeWXwnMfj0R//+Edt2rRJbrdbJpMpgiNMLFOmTNEll1wScu7Tn/60/uu//itCI0ps3/72t7Vq1SotW7ZMknT55Zfr6NGjqqmpIbBcBGpYzsFsNmvhwoWqq6sLnvN6vaqrq9PixYsjOLLE5PP59I1vfEO//OUv9eqrr2rmzJmRHlJCu+mmm/Tee+9pz549waOsrExf+tKXtGfPHsLKBLv66quHtPnv379fM2bMiNCIElt3d7eMxtA/ryaTSV6vN0Ijig/MsJxHdXW1VqxYobKyMpWXl2vjxo1yuVxauXJlpIeWcO6++2698MIL+vWvf62MjAw1NTVJkrKyspSSkhLh0SWejIyMIfVDaWlpys3Npa4oAv7lX/5Fn/nMZ/TII4/o7//+77Vjxw79+Mc/1o9//ONIDy0hff7zn9f3v/99TZ8+XZdeeqnefvttPfHEE/rHf/zHSA8ttvlwXk8++aRv+vTpPrPZ7CsvL/e99dZbkR5SQpI07PGTn/wk0kPDgOuvv9537733RnoYCes3v/mN77LLLvNZLBbfvHnzfD/+8Y8jPaSE5XQ6fffee69v+vTpPqvV6ps1a5bvu9/9rs/tdkd6aDGNfVgAAEDUo4YFAABEPQILAACIegQWAAAQ9QgsAAAg6hFYAABA1COwAACAqEdgAQAAUY/AAgAAoh6BBQAARD0CCwAAiHoEFgAAEPX+H6Dy7aU+v7jOAAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "model = dc.models.CGCNNModel(mode='regression', batch_size=256, learning_rate=0.0008)\n",
        "\n",
        "losses = []\n",
        "\n",
        "for _ in tqdm(range(10), desc=\"Training\"):\n",
        "    loss = model.fit(train_dataset, nb_epoch=1)\n",
        "    losses.append(loss)\n",
        "\n",
        "plt.plot(losses)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "3a41acc6",
      "metadata": {
        "id": "3a41acc6"
      },
      "source": [
        "Once fitting the model, we evaluate the performance of the model using mean squared error metric since it is a regression task. For selection a metric, `dc.metrics.mean_squared_error` function can be used and we evaluate the model by calling `dc.model.evaluate`.`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "id": "57f3a654",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "57f3a654",
        "outputId": "1feb548a-6518-410c-cf1d-4e2bf39b8a7f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Training set score: {'mean_absolute_error': 0.1310973839991837}\n",
            "Test set score: {'mean_absolute_error': 0.13470105945654667}\n"
          ]
        }
      ],
      "source": [
        "metric = dc.metrics.Metric(dc.metrics.mean_absolute_error)\n",
        "print(\"Training set score:\", model.evaluate(train_dataset, [metric], transformers))\n",
        "print(\"Test set score:\", model.evaluate(test_dataset, [metric], transformers))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "9dQ_Ug0iRTwP",
      "metadata": {
        "id": "9dQ_Ug0iRTwP"
      },
      "source": [
        "The original paper achieved a MAE of 0.130 eV/atom on the same dataset (with a 60:20:20 split, instead of the 80:10:10 being used in this tutorial). "
      ]
    },
    {
      "cell_type": "markdown",
      "id": "7379397d",
      "metadata": {
        "id": "7379397d"
      },
      "source": [
        "## Further Reading <a class=\"anchor\" id=\"further-reading\"></a>\n",
        "\n",
        "For further reading on getting started on using machine learning for material science, here are two great resources:\n",
        "- [Getting Started in Material Informatics](https://towardsdatascience.com/getting-started-in-materials-informatics-41ee34d5ccfe)\n",
        "- [A Collection of Open Source Material Informatics Resources](https://github.com/ncfrey/resources)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "df2e9207",
      "metadata": {
        "id": "df2e9207"
      },
      "source": [
        "# Congratulations! Time to join the Community!\n",
        "\n",
        "Congratulations on completing this tutorial notebook! If you enjoyed working through the tutorial, and want to continue working with DeepChem, we encourage you to finish the rest of the tutorials in this series. You can also help the DeepChem community in the following ways:\n",
        "\n",
        "## Star DeepChem on [GitHub](https://github.com/deepchem/deepchem)\n",
        "This helps build awareness of the DeepChem project and the tools for open source drug discovery that we're trying to build.\n",
        "\n",
        "## Join the DeepChem Gitter\n",
        "The DeepChem [Gitter](https://gitter.im/deepchem/Lobby) hosts a number of scientists, developers, and enthusiasts interested in deep learning for the life sciences. Join the conversation!\n"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.13"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}
